Here, I summarized the main point of my internship report. In this project, I used SPARQL endpoint that is very similar with SQL server and PostGIS.
This research project addresses the gap between non-expert users and geoinformation Linked Data technology. In this project, we introduced a geospatial Linked Data application named GeodataVisitor. GeodataVisitor is a Linked data-driven Web GIS application that helps users to integrate and display their datasets in excel format with datasets in Knowledge Graphs using SPARQL services. No deep knowledge of the vocabulary and SPARQL is required for end-users. Users who have a basic understanding of RDF can display their datasets in GeodataVisitor. GeodataVisitor allows users to visit their dataset on a user-friendly map after completing a sequence of tasks through external Web components to transform non-spatial datasets to a standard Linked Geodata. The workflow outputs are the inputs for GeodataVisitor, where SPARQL queries and parametrized values allow users to visualize and integrate their datasets with parametrized datasets retrieved from Knowledge Graphs on the Linked Data Web application.
The first step of the workflow is transforming non-spatial datasets to standard linked data. In the user interface of GeoData Visitor, a link has been provided that allows the user to access the GD Wizard where a user starts with uploading a CSV dataset and configures dataset columns with some vocabularies, and refine the non-spatial data with a spatial dimension. Then, the corresponding BAG or BRT URI is linked using value refinement functionality. For example, postcode and house number can be refined using BAG datasets. The results of this stage are refined RDF and excel files. The property URI and refined URI column are transformed to predicate and object in RDF, respectively, and are published in the PLDN. The user logs in the PLDNenvironment to take an API address and publishes the Linked Data in the PLDN environment. After creating a new SPARQL service for the desired dataset, the username and dataset name allow the user to access their datasets and visualize them on the Web GIS interface (GeodataVisitor). The mentioned steps are for non-expert users who desire to visualize their data on the Web application.
In practice, the mentioned tasks mostly overlap with the Geodata Visitor development. The step after creating a new SPARQL service was different for developers. Column configuration helps developers to write a SPARQL query in the next component (PLDN) and link the external dataset to the dataset on the Knowledge graphs. Indeed, using the property configuration and value refinement functionalities, non-spatial datasets can be interlinked to their corresponding resources in BAG2 or BRT, and finally, Knowledge Graph (kg) where the geometry of data with spatial dimensions are registered. A SPARQL query for each enriched predicate (e.g., sdo: address) and the refined object were manually written to link external datasets to similar resources in Knowledge Graphs. More precisely, when expert users specify a SPARQL query for enriched predicate and object, the SPARQL tool detects and integrates two or more resources with the same object. In the figure, the grey boxes indicate the related components units between the Web applications. The white box shown in the figure represents the source code of GeodataVisitor. In this project, I used two API (SPARQL endpoint). The first API returns the results of integration between user datasets with BAG/BRT and knowledge graph. The result is a JSON list contains user data and Geometry of data. The second API turned information about buildings directly from Knowledge Graphs. The results of two APIs were displayed on both pop-ups and legends. Users can set their preferences to see their desired information the Web application.
PREFIX sdo: https://schema.org/>
PREFIX bag: https://bag2.basisregistraties.overheid.nl/bag/def/>
PREFIX rdf: http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX rdfs:http://www.w3.org/2000/01/rdf-schema#>
PREFIX foaf: http://xmlns.com/foaf/0.1/>
Prefix prov: http://www.w3.org/ns/prov#>
prefix sdo: https://schema.org/>
prefix pldn: https://data.pldn.nl/4b8917/def/>
select distinct
?polygon ?straatadres ?registratie
(strdt(concat('',str(?s),''),
rdf:HTML) as ?polygonLabel)
{
?s
sdo:postalcode ?postcode;
sdo:address ?Nummeraanduiding. # should be linked users
service https://api.labs.kadaster.nl/datasets/kadaster/bag2/services/default/sparql {
?registratie foaf:primaryTopic ?Nummeraanduiding.# correct should be linked BAG2
service https://api.labs.kadaster.nl/datasets/kadaster/kg/services/default/sparql> {
?place
a sdo:Place;
sdo:address ?postadres;
sdo:geo ?bagShape .
?bagShape
a sdo:GeoShape;
sdo:name ?bagShapeNaam;
sdo:polygon ?polygon .
filter(?bagShapeNaam in ("BAG geometrie", "BAG vlakgeometrie")).
?postadres
a sdo:PostalAddress;
sdo:streetAddress ?straatadres;
sdo:postalCode ?postcode;
prov:wasDerivedFrom ?registratie.
}}
}
prefix bif: http://www.openlinksw.com/schemas/bif#>
prefix geo: http://www.opengis.net/ont/geosparql#>
prefix rdf: http://www.w3.org/1999/02/22-rdf-syntax-ns#>
prefix sdo: https://schema.org/>
select distinct ?punt ?bagShape ?place {
bind(strdt(concat('Point(',str(?lat),' ',str(?long),')'),geo:wktLiteral) as ?punt)
# Step 1: Find all places for which one geometry matches.
# Any geometry can be used to match: BAG, BGT, BRT.
{
select
?place0
(group_concat(distinct concat(
replace(str(?type),".*[\\/#]",""),
{
bind(strdt(concat('Point(',str(?lat),' ',str(?long),')'),geo:wktLiteral) as ?punt0)
?place0
a sdo:Place;
sdo:additionalType ?type;
sdo:geo ?geoMatch.
?geoMatch sdo:polygon ?polygon.
filter(bif:st_within(?punt0, ?polygon))
}
group by ?place0
}
# Step 2: For each matched place, retrieve all geometries (BAG, BGT, BRT).
bind(strdt(concat('Point(',str(?lat),' ',str(?long),')'),geo:wktLiteral) as ?punt0)
?place0 sdo:name ?name.
optional {
?place0 sdo:geo ?geoBag.
?geoBag sdo:name ?geoBagName; sdo:polygon ?bagShape0.
filter(?geoBagName in ("BAG geometrie", "BAG vlakgeometrie") && bif:st_within(?punt0, ?bagShape0))
}
bind(coalesce(?bagShape0) as ?bagShape)
}
limit 100
Sample Javascript codes for getting APIs and display map in the leaflet
//@ts-check
import * as wellKnown from "wellknown";// Well-known text (WKT) is a text markup language for representing vector geometry objects.
import * as turf from "@turf/turf";
import _ from "lodash";
//@Select a Sparql query
//export const apiAddress = "https://api.labs.kadaster.nl/queries/BibiMaryam-SajjadianJaghargh/bag-postcode-1/run"
export const apiAddress = "https://api.labs.kadaster.nl/queries/BibiMaryam-SajjadianJaghargh/geo-object/run"
export interface SparqlResults {
head: Head;
results: {
bindings: Binding[];
};
}
export interface Head {
vars: string[];
}
export interface Binding {
[varname: string]: BindingValue;
}
export type BindingValue =
| {
type: "uri";
value: string;
}
| {
type: number;
value: string
}
| {
type: "literal";
value: string
};
/**
* Convert the sparql json results of user API into a Result.js array
*/
//
export async function searchResourcesDescriptions(apiAddress: string, res:SparqlResults) {
let propNames = res.head.vars
return Promise.all(res.results.bindings.map(async b => {
let geoJson = null
let properties: any = {}
for(let prop of propNames) {
if(prop === 'polygon') {
geoJson = wellKnown.parse(b[prop].value)
}
else if(prop === 'polygonLabel') {
properties['User Dataset'] = (/ href=\"(?href>.*)\" /.exec(b[prop].value).groups || {}).href
}
else if(b[prop].type === 'uri') {
properties[prop] = b[prop].value
}
}
let coords = turf.center(geoJson).geometry.coordinates
console.log(coords)
let data
try {
data = await fetch(`${apiAddress}?lat=${coords[0]}&long=${coords[1]}`).then(result => result.json())
.then(result => result[0])
console.log("Input and data: ");
console.log(coords[0]);
console.log(coords[1]);
console.log(data);
delete data.bagShape
delete data.punt
delete data.BagShape0
} catch (error) {}
return {
sub: 'registratie',
geo: geoJson,
...properties,
...(data || {})
};
}))
}
/**
* Get the text search result from user's api
* @param api
* @returns
*/
// new case
export async function searchQuery(username:string, datasetName: string): Promise {
let api = `https://api.data.pldn.nl/datasets/${username}/${datasetName}/services/${datasetName}/sparql`
const sparqlQuery = `PREFIX sdo: https://schema.org/>
PREFIX bag: https://bag2.basisregistraties.overheid.nl/bag/def/>
PREFIX rdf: http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX rdfs: http://www.w3.org/2000/01/rdf-schema#>
PREFIX foaf: http://xmlns.com/foaf/0.1/>
Prefix prov: http://www.w3.org/ns/prov#>
prefix sdo: https://schema.org/>
prefix pldn: https://data.pldn.nl/4b8917/def/>
select distinct
?polygon ?straatadres ?BAG2
(strdt(concat('a href="https://data.pldn.nl/${username}/${datasetName}/browser?resource=',encode_for_uri(?s),'" target="_link">',str(?s),''),
rdf:HTML) as ?polygonLabel)
{
?s
sdo:postalcode ?postcode;
sdo:address ?Nummeraanduiding. # should be linked users
service https://api.labs.kadaster.nl/datasets/kadaster/bag2/services/default/sparql> {
?BAG2 foaf:primaryTopic ?Nummeraanduiding.# correct should be linked BAG2
service https://api.labs.kadaster.nl/datasets/kadaster/kg/services/default/sparql> {
?place
a sdo:Place;
sdo:address ?postadres;
sdo:geo ?bagShape .
?bagShape
a sdo:GeoShape;
sdo:name ?bagShapeNaam;
sdo:polygon ?polygon .
filter(?bagShapeNaam in ("BAG geometrie", "BAG vlakgeometrie")).
?postadres
a sdo:PostalAddress;
sdo:streetAddress ?straatadres;
sdo:postalCode ?postcode;
prov:wasDerivedFrom ?BAG2.
}}
} limit 30`
//if user refined postalcode = sdo:postalcode ?Nummeraanduiding; for example for postaddreshous dataset
const result = await fetch(api, {
method: "POST",
headers: {
"Content-Type": "application/sparql-query",
Accept: "application/sparql-results+json"
},
body: sparqlQuery
});
if (result.status > 300) {
throw new Error("Request with response " + result.status);
}
return result.json();
}
/*
* Libs
*/
import * as wellKnown from "wellknown";
import L, { LatLng, LatLngBounds } from "leaflet";
import * as turf from "@turf/turf";
const inside = require("point-in-geopolygon");
import _ from "lodash";
import "leaflet.markercluster";
import * as GeoJson from "geojson";
import "react-toastify/dist/ReactToastify.css";
import "./styles.scss";
import * as Reducer from "./reducer";
import { objectToGeojson, getAllObjectsAsFeature } from "./helpers/utils";
import { DefaultIcon, Icons } from "./components/Icons";
/*
* Assets
*/
import "leaflet.markercluster/dist/MarkerCluster.css";
import "leaflet.markercluster/dist/MarkerCluster.Default.css";
let map: L.Map;
let geoJsonLayer: L.GeoJSON;
let markerGroup: any;
export type FeatureProperties = Reducer.SingleObject;
export type GeojsonFeature = GeoJson.Feature
GeoJson.Geometry,
FeatureProperties
>;
/*
* @param opts
*/
export function init(opts: {
onContextSearch: (context: Reducer.CoordinateQuery) => void;
onZoomChange: (zoomLevel: number) => void;
onClick: (el: Reducer.SingleObject) => void;
onLayersClick: (info: Reducer.State["clickedLayer"]) => void;
}) {
//Define layers (BasedMap, ArialPhoto, OpenStreetMaps )
const brtKaart = L.tileLayer(
"https://geodata.nationaalgeoregister.nl/tiles/service/wmts/brtachtergrondkaart/EPSG:3857/{z}/{x}/{y}.png",
{
attribution:
'Kaartgegevens © a href="https://www.kadaster.nl/" target="_blank" rel = "noreferrer noopener">Kadaster | Verbeter de kaart ',
}
);
const luchtfotorgb = L.tileLayer(
"https://service.pdok.nl/hwh/luchtfotorgb/wmts/v1_0/2020_ortho25/EPSG:3857/{z}/{x}/{y}.jpeg",
{
attribution:
'Landelijke Voorziening Beeldmateriaal © a href="https://www.pdok.nl/" target="_blank" rel = "noreferrer noopener">PDOK ',
}
);
const OSM = L.tileLayer(
"https://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png",
{
attribution:
'© a href="https://www.openstreetmap.org/copyright">OpenStreetMap/a> contributors',
}
);
//Group the basemap layers...here changed
let baseMaps = {
"Luchtfoto's": luchtfotorgb,
"Open Street Map": OSM,
"BRT Achtergrondkaart": brtKaart,
};
//Define the map
map = L.map("map", {
minZoom: 8,
center: [52.20936, 5.2],
zoom: 8,
maxBounds: [
[56, 10],
[49, 0],
],
layers: [brtKaart, luchtfotorgb, OSM],
});
//Add layer control(select basemap) for the last version
L.control.layers(baseMaps).addTo(map);
//When you click on the card, all locations get back arround.
// Send request
map.on("contextmenu", async (e) => {
let latLong = (e as any).latlng;
let data: any = undefined;
let bagShape: any = undefined;
try {
if((window as any).apiAddress) {
data = await fetch(`${(window as any).apiAddress}?lat=${latLong.lng}&long=${latLong.lat}`)
.then((result) => result.json())
.then((result) => result[0]);
//console.log("right click");
//console.log(data);
}
if (!data) {
return;
}
bagShape = data.bagShape;
delete data.bagShape;
delete data.punt;
} catch (error) {}
if (data) {
markerGroup.clearLayers();
geoJsonLayer.clearLayers();
let geoJson = wellKnown.parse(bagShape);
geoJsonLayer.addData([
{ type: "Feature", geometry: geoJson, properties: data, org: true },
] as any);
}
});
/**
* The function that the card calls every time it want to add a marker.
**/
const addMarker = (feature: GeojsonFeature, latlng: L.LatLng): any => {
// Create a marker
let marker = L.marker(latlng);
marker.feature = {
type: "Feature",
geometry: { type: "Point", coordinates: [latlng.lat, latlng.lng] },
properties: feature.properties,
};
markerGroup.addLayer(marker);
//Method that are called to open the marker
let onHover = function(this: L.Marker) {
this.openPopup();
this.setIcon(Icons);
}.bind(marker);
//Method that is called to close the marker
let onHoverOff = function(this: L.Marker) {
this.closePopup();
this.setIcon(DefaultIcon);
}.bind(marker);
//When you click on it Go to that marker
marker.on(
"click",
() => {
opts.onClick(feature.properties as any);
},
marker.openPopup()
);
// When you cross the marker Let the pop up
marker.on("mouseover", onHover);
// When you leave it from it
marker.on("mouseout", onHoverOff);
return marker;
};
const addMarkerForNonPoint = (feature: GeojsonFeature, latlng: L.LatLng) => {
//Create a marker
let marker = L.marker(latlng);
marker.feature = {
type: "Feature",
geometry: { type: "Point", coordinates: [latlng.lat, latlng.lng] },
properties: feature.properties,
};
//console.log("popup");
//console.log(latlng.lat);
//console.log(latlng.lng);
//console.log(feature.properties.place);
// this is the popup and the html that will appear.
marker.bindPopup(
`div class = "marker">
${
(feature as any).org
? ""
: `b>a href=
https://data.pldn.nl/
target="_blank" >PLDN/a>
/b> `
}
br/>
${Object.keys(feature.properties)
.map((k) => {
function makeDate(date: Date) {
return new Date(date.getTime());
}
const newDate = new Date(); //
const today2 = makeDate(newDate);
if (k === "sub") return "";
if (k === "geo") return "Date: "+today2;
if (
[
"BAG2",
"bag",
"brt",
"bgt",
"woonplaats",
"verblijfsobjectStatus",
"VerblijfsobjectStatus",
"User Dataset",
"nummeraanduiding",
"Verblijfsobject",
].includes(k)
) {
return `a href="${
feature.properties[k]
}" target="_blank">${k.toUpperCase()}/a>`;
}
return `b>${k[0].toUpperCase() + k.slice(1)}: ${
feature.properties[k]
}/b>`;
})
.join("br/>")}
br/>
div>
`,
{
autoPan: false,
closeButton: false,
}
);
//Method that are called to open the marker
let onHover = function(this: L.Marker) {
this.openPopup();
this.setIcon(Icons); //icons??? not default
}.bind(marker);
//Method that is called to close the marker
let onHoverOff = function(this: L.Marker) {
this.setIcon(DefaultIcon);
}.bind(marker);
//When you cross the marker Let the pop up see
marker.on("mouseover", onHover);
//When you leave it from it
marker.on("mouseout", onHoverOff);
//When you click on it Go to that marker
marker.on("click", function(this: L.Marker) {
opts.onClick(feature.properties);
this.openPopup();
});
marker.on("dblclick", () => {
let bbox = turf.bbox(feature as any);
map.fitBounds(
new LatLngBounds(
new LatLng(bbox[1], bbox[0]),
new LatLng(bbox[3], bbox[2])
)
);
});
return marker;
};
/**
* Called every time a geojson object is drawn.
**/
const handleGeoJsonLayerDrawing = (
feature: GeojsonFeature,
layer: L.Layer
) => {
if (feature.geometry.type === "Point") return;
//First find the center
let latLong = getCenterGeoJson(feature);
//console.log("feature");
//console.log(feature.properties.place);
//On this center add a marker
markerGroup.addLayer(addMarkerForNonPoint(feature, latLong));
//If you click on it there then
layer.on("click", (e: any) => {
//Check if there are several layers
let contains = getAllGeoJsonObjectContainingPoint(
e.latlng.lng,
e.latlng.lat
);
//If only one is low
if (contains.length < 2) {
opts.onClick(feature.properties as any);
} else {
opts.onLayersClick({
x: e.originalEvent.pageX,
y: e.originalEvent.pageY,
values: contains.reverse().map((res) => res.properties),
});
}
});
};
geoJsonLayer = L.geoJSON([] as any, {
onEachFeature: handleGeoJsonLayerDrawing,
pointToLayer: addMarker as any,
style: {
color: "orange",
opacity: 0.4,
},
}).addTo(map);
// the group for the markers
markerGroup = (L as any).markerClusterGroup({
showCoverageOnHover: false,
});
map.addLayer(markerGroup);
//This is for mobile application.If dragged then closes the context menu.
map.on("dragstart", () => {});
map.on("zoomend" as any, () => {
opts.onZoomChange(map.getZoom());
});
}
export function closePopup() {
if (map) map.closePopup();
}
export function centerMap() {
map.setView([52.20936, 5.2], 8);
}
export function updateMap(opts: {
selectedObject?: Reducer.SingleObject;
searchResults?: Reducer.State["searchResults"];
properties?: Reducer.State["properties"];
updateZoom: boolean;
}) {
map.closePopup();
markerGroup.clearLayers();
geoJsonLayer.clearLayers();
let points = opts.searchResults.map((s) => {
let newS = {} as Reducer.SingleObject;
newS.geo = s.geo;
newS.sub = s.sub;
for (let [k, v] of Object.entries(opts.properties)) {
if (v) {
newS[k] = s[k];
}
}
return newS;
});
// If there is a clicking result, render only this one
if (opts.selectedObject) {
geoJsonLayer.addData(objectToGeojson(opts.selectedObject));
map.fitBounds(L.featureGroup([geoJsonLayer, markerGroup]).getBounds());
} else if (points.length) {
let features = getAllObjectsAsFeature(points) as any;
geoJsonLayer.addData(features);
map.fitBounds(L.featureGroup([geoJsonLayer, markerGroup]).getBounds());
} else if (opts.updateZoom) {
centerMap();
}
}
export function toggleClustering(toggle: boolean) {
if (toggle) {
map.removeLayer(markerGroup);
markerGroup = (L as any).markerClusterGroup({
showCoverageOnHover: false,
});
map.addLayer(markerGroup);
} else {
map.removeLayer(markerGroup);
markerGroup = L.featureGroup().addTo(map);
}
}
const getAllFeaturesFromLeaflet = () => {
return geoJsonLayer
.getLayers()
.map((l: any) => l.feature) as GeojsonFeature[];
};
export function findMarkerByUrl(registratie: string) {
return markerGroup.getLayers().find((l: any) => {
const feature: GeojsonFeature = l.feature;
return feature.properties.sub === registratie;
});
}
/**
* Get all Geojson objects that are in the results holder where this item is in.
*/
const getAllGeoJsonObjectContainingPoint = (lng: number, lat: number) => {
return getAllFeaturesFromLeaflet().filter((res) => {
if (res.geometry.type !== "MultiPolygon" && res.geometry.type !== "Polygon")
return false;
let col = { type: "FeatureCollection", features: [res] };
//Filter, when ER -1 exceeds, the point is not in the polygon.
return inside.feature(col, [lng, lat]) !== -1;
});
};
const getCenterGeoJson = (geojson: any): L.LatLng => {
let centroid = turf.center(geojson);
//maak er een geojson en feature van. = make it a geojson and feature.
let geoJsonFeature = geojson.geometry
? geojson
: { type: "Feature", geometry: geojson };
geojson = geojson.geometry ? geojson.geometry : geojson;
//Multipolygon werkt niet met turf.booleanContains. = Multipolygon does not work with turf.booleanContains.
if (geojson.type !== "MultiPolygon") {
//als deze niet in het geojson object ligt, gebruik dan de centroid = if it is not in the geojson object, use the centroid
if (!turf.booleanContains(geoJsonFeature, centroid)) {
centroid = turf.centroid(geoJsonFeature);
}
//anders gebruik point on feature = otherwise use point on feature
if (!turf.booleanContains(geojson, centroid)) {
centroid = turf.pointOnFeature(geojson);
}
} else {
//gebruik inside voor multipolygon om te controlleren. = use inside for multipolygon to check.
let lon = centroid.geometry.coordinates[0];
let lat = centroid.geometry.coordinates[1];
let col = { type: "FeatureCollection", features: [geoJsonFeature] };
let isInside = inside.feature(col, [lon, lat]) !== -1;
if (!isInside) {
centroid = turf.centroid(geojson);
}
lon = centroid.geometry.coordinates[0];
lat = centroid.geometry.coordinates[1];
col = { type: "FeatureCollection", features: [geoJsonFeature] };
isInside = inside.feature(col, [lon, lat]) !== -1;
if (!isInside) {
centroid = turf.pointOnFeature(geojson);
}
}
//Get the bar and lung
let lon = centroid.geometry.coordinates[0];
let lat = centroid.geometry.coordinates[1];
return L.latLng(lat, lon);
};
